Methodological support of the working group in predicting the results of the classification expertise

Burkov E.A., Paderno P.I., Sattorov F.E., Tolkacheva E.A. Methodological support of the working group in predicting the results of the classification expertise. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2021, vol. 21, no. 3, pp. 426–432 (in Russian). doi: 10.17586/2226-1494-2021-21-3-426-432

The paper considers the specifics of the working group activity in preparation of expertise classification and the features of approaches for the choice of expert assessments and expert selection methods. The analysis focuses on potentially weak (illegibly) formalized customer requirements, which are quite typical for expertise classification. The used methods suffer from many drawbacks, which make them practically inapplicable in terms of planning, preparing and predicting the possible expertise classification reliability. The authors developed a new approach for predicting the reliability of the expertise classification at the stage of its preparation that depends on the reliability of the proposed expert group. The approach involves a probabilistic representation of the possible results of the work (namely, classification) of particular experts. The authors propose a number of probabilistic models (probabilistic matrices), which reflect the reliability (correctness) of the classification of certain objects both at the level of particular experts and at the level of entire expertise results. The set of procedures developed for a random group of experts allows obtaining probabilistic characteristics of the objects classification correctness when the group works as a part of an expert commission. The proposed approach can be used as a tool for working groups, which not only simplifies the process of an expert group selection, but also allows predicting the reliability of possible results and thereby makes it possible to take measures in advance in order to meet customer requirements. This approach can serve as a methodological basis for automating the problem solution in the expert selection process for the expertise classification at the level of its preparation, depending on customer requirements (restrictions). The proposed models and procedures will improve the efficiency of expertise classification, as well as save time for its preparation.

1. Litvak B.G. Development of Management Solutions. Moscow, Delo Publ., 2004, 392 p. (in Russian)

2. Burkov E.A., Paderno P.I., Paharkov G.N. Expertise: System problems and solutions in selecting medical equipment. Biotekhnosfera, 2010, no. 2(8), pp. 6–14. (in Russian)

3. Saaty Th.L., Kearns K.P. Analytical Planning: The Organization of Systems. RWS Publications, 1985, 208 p.

4. Saaty Th.L. Decision Making with Dependence and Feedback: The Analytic Network Process: the Organization and Prioritization of Complexity. RWS Publications, 1996, 370 p.

5. Paderno P.I. Method of combination experts' opinion in group using hierarchy analysis method. News of the Saint Petersburg State Forest Technical Academy, 2009, no. 189, pp. 238–245. (in Russian)

6. Pаderno P.I. Combination opinions of commissions of experts at the estimation of the importance of indicators. News of the Saint Petersburg State Forest Technical Academy, 2010, no. 190, pp. 207–211. (in Russian)

7. Dutova E.D., Nasarenko N.A., Paderno P.I. Analysis of the influence of transformation and integration technology of the expert evaluations on the result. Proc. 19th International Conference on Soft Computing and Measurements (SCM), 2016, pp. 21–24. doi: 10.1109/SCM.2016.7519671

8. Naychenko M.V. Ergonomic support for the creation of human-machine systems. Biotekhnosfera, 2015, no. 1(37), pp. 10–13. (in Russian)

9. Burkov E.A., Karpachevskiy A.V., Paderno P.J. Determination of experts competence based on their performance in expertises. Proceedings of Saint Petersburg Electrotechnical University, 2011, no. 10, pp. 38–44. (in Russian)

10. Burkov E.A., Paderno P.I. An approach to forming an expert group as a task of discrete optimization. Journal Biomedical Radioelectronics, 2010, no. 5, pp. 48–51. (in Russian)

11. Burkov E.A., Paderno P.I., Siryk O.E., Lavrov E.A., Pasko N.B. Analysis of impact of marginal expert assessments on integrated expert assessment. Proc. 23rd International Conference on Soft Computing and Measurements (SCM), 2020, pp. 14–17. doi: 10.1109/SCM50615.2020.9198772

12. Danelyan T.Y. Formal methods of expert estimations. Economics, Statistics and Informatics. Vestnik UMO, 2015, no. 1, pp. 183–187. (in Russain)

13. Paderno P.I., Burkov E.A., Lavrov E.A. Issues of organization of expertise and problems of expert assessments. Journal of Physics: Conference Series, 2020, vol. 1703, pp. 012047. doi: 10.1088/1742-6596/1703/1/012047

14. Maslennikov E.V. Features of selection of experts. Sociology, 2010, no. 2, pp. 82–93. (in Russian)

15. Paderno P.I. The system of intellectual support examination. Proceedings of Saint Petersburg Electrotechnical University, 2005, no. 9, pp. 3–8. (in Russian)

16. Burkov E.A., Lyubkin P.L., Paderno P.I. Intellectual systems – the future of expert assessment. Proc. 20th IEEE International Conference on Soft Computing and Measurements (SCM), 2017, pp. 34–36. doi: 10.1109/SCM.2017.7970487

17. Nazarenko N.A., Paderno P.I., Sattorov F.E. Estimation method for logical difficulty and stereotype of operators' activity algorithms. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2019, vol. 19, no. 4, pp. 740–746. (in Russian). doi: 10.17586/2226-1494-2019-19-4-740-746

18. Makarchenko M., Borisova I., Sattorov F. Approach changing into organization processes and personnel management in context of digitalization. IOP Conference Series: Materials Science and Engineering, 2020, vol. 940, pp. 012097. doi: 10.1088/1757-899X/940/1/012097

19. Burkov E.A., Nazarenko N.A., Nasser S.S.S., Paderno P.I. Analysis of correctness of linear transformations of expert estimates. Proc. 22th IEEE International Conference on Soft Computing and Measurements (SCM), 2019, pp. 29–32. doi: 10.1109/SCM.2019.8903758

20. Andreevsky E.V., Burkov E.A., Nazarenko N.A., Paderno P.J. The methods of evaluating the effectiveness of strategies professional psychological selection. Proceedings of Saint Petersburg Electrotechnical University, 2015, no. 7, pp. 19–25. (in Russian)

21. Andreevskiy E.V., Paderno P.I. Structure of automated information support system for professional selection of the personnel securing nuclear facility and its place in the structure of automated industrial management system. Systems. Methods. Technologies, 2016, no. 2(30), pp. 109–113. (in Russian). doi: 10.18324/2077-5415-2016-2-109-113

22. Paderno P.I., Andreevskii E.V. A program for organizing professional selection. Certificate of state registration of a computer program № RU2017617164, 30 June 2017. (in Russian)

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License